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Abstract. Counting problems such as determining how many bit strings satisfy 
a given Boolean logic formula are notoriously hard. In many cases, even getting 
an approximate count is difficult. Here we propose that entanglement, a common 
concept in quantum information theory, may serve as a telltale of the difficulty 
of counting exactly or approximately. We quantify entanglement by using Renyi 
entropies S( q \ which we define by bipartitioning the logic variables of a generic 
satisfiability problem. We conjecture that S^ 9 ^ ' provides information about the 
difficulty of counting solutions exactly, while S^ q> ^ indicates the possibility of doing 
an efficient approximate counting. We test this conjecture by employing a matrix 
computing scheme to numerically solve #2SAT problems for a large number of 
uniformly distributed instances. We find that all Renyi entropies scale linearly with the 
number of variables in the case of the #2SAT problem; this is consistent with the fact 
that neither exact nor approximate efficient algorithms are known for this problem. 
However, for the negated (disjunctive) form of the problem, S*^^ ) scales linearly while 
tends to zero when the number of variables is large. These results are consistent 
with the existence of fully polynomial-time randomized approximate algorithms for 
counting solutions of disjunctive normal forms and suggests that efficient algorithms 
for the conjunctive normal form may not exist. 
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1. Introduction 



Satisfiability (SAT) is an NP-complete problem that aims at deciding whether there 
is an n-bit string input that satisfies a Boolean logic formula [U |2] . An example of a 
satisfiability problem is Circuit Satisfiability, or CSAT, where a circuit is built with a 
number of gates that is polynomial in n. While the cost of testing whether a given 
n-bit string satisfies the circuit is polynomial, finding whether the circuit is satisfiable 
is a hard problem, and counting the number of satisfying inputs is even harder. The 
problem of counting satisfying solutions of SAT is known as #SAT [31 H]. 
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Table 1. Summary of the results for the #2SAT problem. 
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The difference in complexity between finding and counting solutions [3] is clearly 
illustrated in the case of 2SAT, the problem of satisfiability of logic formulas built using 
Boolean expressions involving exactly two literals (or bits). While the logic expression 
for a problem in 2SAT can be determined to be satisfiable or not in polynomial time 
(2SAT G P), it is believed that counting the number of all satisfying solutions (when they 
exist) cannot be done efficiently. Indeed, #2SAT is a problem in the class #P-complete, 
which is also the same class containing #3SAT, although 3SAT is in NP-complete. 
In other words, even though it is much easier to solve 2SAT than 3SAT, counting 
the satisfying solutions is as difficult in one problem as in the other. Algorithms for 
counting exactly the solutions of the #SAT problem exist (see for instance (6j [7]), but 
they require a number of operations that scales exponentially with n. No polynomial 
or even subexponential algorithm is known. 

The fact that a non-exponential algorithm for counting solutions of a SAT problem 
is still unknown raises the following question: Is there an entropic principle that limits 
the efficiency of large-scale counting machines, much as there is one that limits the 
efficiency of thermal engines? Here we propose the Renyi entanglement entropies as 
means to quantify the difficulty of counting problems. We test the idea explicitly for 
the case of #2SAT stated in conjunctive normal form (CNF) and its negation, which is 
stated in disjunctive normal form (DNF). 

The entanglement entropy, a concept of much use in quantum information theory, 
differs from the thermodynamic entropy. In the case of SAT problems, the usual entropy 
S tells us about the number of solutions Z, i.e. S = log 2 Z. For example, if instead 
of counting the number solutions one were asked to present all solutions, it would take 
a time 0{2 S ) to do so. But counting the number of solutions is a bit easier, in the 
sense that one could "compress" the information needed to do the counting without 
presenting all solutions. The degree of compression of the information needed to do the 
counting is what we relate below to the Renyi entanglement entropies . 

In this article, we show rather generically how one can define the Renyi entropies 

associated to problems of Boolean expression satisfiability. We then focus on the 
particular case of random #2SAT problems, compute the Renyi entropies S^ ^ and 
S^ q=2 \ and, in certain cases, S^ 9- * 1 ) as well. 
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Figure 1. Example of a bipartition of a system of 14 Boolean variables into two sets: 
A (dark blue) and B (light red). The dashed lines represent 2- literal clauses entering 
in the Boolean expression of a 2SAT problem. 

We conjecture that the Renyi entropy S^ ^ provides information on the difficulty 
of counting solutions exactly, while the Renyi entanglement entropies S^ q>0 ^ tell us 
about the possibility of doing efficient (polynomial-time) approximate counting. More 
precisely, we conjecture that what determines if the exact counting can or cannot be 
done efficiently is whether S 1 ^ ) scales with n slower or faster than log 2 n. Similarly, 
what determines whether an approximate counting can or cannot be done efficiently is 
whether S^ q>0 ^ scales with n slower or faster than log 2 n. We then test this conjecture in 
the case of #2SAT and its negated version using the entropies we obtained numerically 
for random instances of the problem. We find that in these cases the entropies give 
the correct prediction for whether efficient algorithms should exist or (likely) not. Our 
results are summarized in Table 1. 

2. Renyi entropies and t^SAT 

Let us start by considering a set of n binary variables {xj = 0, 1}j=i „ )n , and a binary 
weight W(x) = W(xi, . . . , x n ) = or 1, depending on whether the string of binary 
variables x = X\ x 2 . . . x n satisfies or not a given Boolean expression. With the weights 
W(x), we define the vector 

\W)= W(x 1 ,...,x n )\x 1 ...x n ), (1) 

a; n ,...,zi=0,l 

where \xi x% . . . x n ) = \x) denotes a particular configuration of this binary system. 
Next, we construct the (unnormalized) density matrix 

e = \W)(W\ = ^W{x)W(x')\x)(x'\ . (2) 

x,x' 

The number of strings x satisfying the given Boolean expression is given by 

Z = tr q = W(x) 2 = W(x) , (3) 

X X 

where we used that W(x) = 0, 1. If one wishes, a normalized density matrix p = g/Z 
can also be constructed. 
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The entanglement Renyi entropies are constructed from reduced density matrices 
after dividing the system into subsystems A and B (see figured] for an example). Let 
system A be comprised of bits 1 to I, and system B of bits I + 1 to n: x A = x\x 2 . . . xi 
and xb = xi + i...x n , so that x = x a xb- Next, construct a 2 l x 2 n ~ l matrix 
W XA ,x B = W(xaXb) out of the list of weights W(x). One then defines the reduced 
density matrix 

Qa = tr B Q 

= E \ J2 W ( x a x b)W(x' a x b )\\xa)Wa\ 

x a> x 'a \ x b ) 

= E [WW T ]x A ,x> A \x A ){x' A \ . (4) 



The Renyi entanglement entropies j8] are given by 

(5) 



<? (9) - 1 W 

AB - j _ 10 §2 



tr^ ^ 



It follows from the cyclicity of the trace that S AB = S BA , thus the entropies are 
independent of the order of the traces (i.e. of which of A or B is traced out first). 

The entanglement entropies depend only on the singular values resulting from the 
decomposition W = U A V T , where U is an orthogonal 2 l x 2 l matrix, V is an orthogonal 
2 n ~ l x 2 n ~ l matrix, and A is a 2 l x 2 n ~ l rectangular diagonal matrix with elements 
Afc, k = 1, . . . , d — min(2 ; , 2 n ~ l ). The entanglement entropies are given in terms of these 
singular values by 



- lop- 

The number of satisfying solutions Z is also linked to the singular values: 



(6) 



z = J2 x l- ( 7 ) 

k=l 

The number of singular values depends on how the system is partitioned. But 
the partition into two systems is just one step of many: one can further recursively 
split systems A and B each into two, A X) A 2 , B ly and B 2 , and so on. This is a way to 
construct a representation of the W(x) as a matrix product state (MPS). The partitions 
that lead to the largest number of singular values are those when systems A and B are 
of the same order, so we shall focus on this case for the discussion that follows. 

To get the exact value of Z, one must sum over all the singular values. There are 

possibly d of them, but many can be zero. So how many are there that are non-zero? 

The number r of non-zero singular values are related to the Renyi entropy with q — > 0, 

\ 2c > 



which counts all the non-zero all these weighted with the same factor liuu^o A 2<? 



One finds log 2 r = S AB °^ . 

The rank r sets the size of the matrices that can represent W(x) as a MPS. 
Therefore, the quantity S^ " 1 is basically a measure of the amount of resources needed 
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Figure 2. Renyi entropies for the #2SAT problem as functions of the number of 
clauses for n = 20. The data points are averaged over 4000 realizations. The graph 
provides a comparison between exhaustive enumeration of solutions and the matrix 
product states method for the cases q = and q = 2. The case q = 1 obtained by 
exhaustive enumeration is also shown. 



to compress the information in all the solutions of the SAT problem. One can show [9] 
that r < min(2 / ,2 n - i ,Z), and thus S%g 0) < S = \og 2 Z. Because S%g 0) < S, the 
complexity of counting is smaller than that of listing the solutions, for counting could 
in principle be done by working with the compressed information without the need to 
expand it. 

3. Numerical computations of for the t^2SAT problem 

Let us henceforth concentrate on calculating the Renyi entropies for the specific case 
of #2SAT. We consider random 2SAT problems, where m clauses are drawn uniformly 
among n(n — l)/2 bit pairs, and among the four possible clauses that involve an OR 
and two literals (each of which can be negated or not). We then compute the entropies 
'Sab* ' 1 an d . Data for systems of size up to n = 20 can be easily obtained from the 
weights W(x) by exhaustive enumeration of all possible inputs x (see figure EJ); however, 
for larger sizes we deploy a matrix computing scheme based on the method of pU] 



(details of the method are provided in Appendix A and Appendix B). The results from 



the matrix computing method for q = and q = 2 were compared against the results 
of exhaustive enumeration of solutions for the same random problem, realization by 
realization, and found to match each other within the relative error margin controlled 
by the threshold used to distinguish the smallest non-zero singular value from zero 
(typically, 10~ 10 ). In figure [21 we present results using both methods for n = 20 after 
averaging over 4000 realizations. 

In figure |3k we show how the entropies per bit s^ = S^ ^ /n and s^ = S^ 2 ^ jn 
vary as we increase the ratio between the number of gates and the number of bits, 
a = m/n. The numerical data was averaged over 4000 realizations of random 2SAT 
problems. The very weak dependence of the entropies per bit on n lead us to conclude 
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Figure 3. (a) Dependence of two Renyi entropies per bit, s(°) = S^^/n and 
s' 2 ) = S^ q=2 ^ /n, with the ratio a — n/m. The data shows that the entropies per bit 
are very weakly dependent on n for a < 1. The matrix product state method was 
employed in the simulations and a total of 4000 realizations were used for each data 
set. (b) The relative deviation s*- 2 -* (n) / (28) — 1 as a function of a, showing in more 
detail the weak dependence of with n when a < 1. The strong deviation seen 
when a > 1 indicates the onset of the 2SAT phase transition at a = 1. The lines are 
guides to the eye. 



that both S^ B and S^g scale as the "volume", i.e. the entropies are very closely 
proportional to the system size n up to a — 1. The deviation from this linear behavior 
is very small, less than 3%, as shown in figure [3b, and can be attributed to the finite 
number of bits in our simulations. A finite-scaling analysis indicates that the entropies 
per bit saturate to a finite value as n — > oo when a < 1. We note that the value 
a = 1 marks the onset of a well-known phase transition in the random 2SAT problem 
PH Q21 QSJ: for a > 1, one expects S^/n as n — > oo (i.e., the number of solutions 
goes to zero when m > n). This is consistent with the growing deviation from the linear 
behavior that we observe in figure [3b beyond a = 1. 

From the perspective of the matrix computing technique we employed in our 
simulations, the difficulty in counting solutions of a given realization of an m-clause 
#2SAT problem is not directly related to the value of computed after the mth 
clauses. Instead, the difficulty is measured by the maximum value that reaches 
as the logic gates are applied, and this maximum can be reached at an intermediate 
number of clauses m* , before all m clauses are enforced. Typically, Renyi entropies 
reach a maximum value around a value m* m n and quickly decay beyond that point 
(see figure [2]). Nevertheless, for all values of m, the averaged Renyi entropies for CNF 
always scale linearly with n, provided that a < 1. 
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4. The negation version of the #2SAT 

Next, we compare and contrast the entanglement entropies computed for the 2SAT 
problems with those for the negation of the same problems. More precisely, let us 
consider the Boolean expressions W(x) = if W(x) = 1 and W(x) = 1 if W(x) = 0. 

The singular values obtained from the weights W(x) are intimately correlated to 
those derived from the weights W(x). We find empirically that the relation for large 



n should be as follows (see Appendix C for a heuristic argument) in the case when the 
number of satisfying solutions is less than the number of non-solutions, i.e. Z < 2 n /2. 
For each and every realization of the problem, if there are r non-zero singular values A^, 
k = 1, . . . ,r, for the 2SAT problem with weights W(x), then there are r + 1 singular 
non-zero singular values Xf., k = 0, . . . ,r, for the negated problem with weights W(x). 
The relation between these two sets of singular values, which should hold in the large n 
limit, is 

Aq = 2 n - 2Z (8a) 

Xl = Xl k = l,...,r. (8b) 

Notice that Z = Y^k=o ^fc = ^" — Z, which is the number of solutions of the negated 
problem. To support our claim that the singular values of the two problems are 
related according to (I8al86[) . we plot in figure H] the pairs of singular values (A|, Af), 
for k = l,...,r. The data displayed are for n = 20 and 1000 realizations of the 
random 2SAT problem and its negation. We also computed the relative deviation 
e = | Ag — (2 n — 2Z)\/ (2 n — 2Z) for the highest singular value, and averaged this deviation 
over the 1000 realizations. We find that e avc < 0.3%. We conclude that even for n as 
small as 20 the relations in (18 al!8 a\i hold rather well. 

The relationship between the singular values for W and W yields, via a 
connection between the respective Renyi entropies: 

2(i-^i ? i = ( Z /Z) q 2^-^b + (1 - Z/Z) q . (9) 

In particular 

2 S AB = 2 s ab + 1 j (10) 



as expected since there is one more singular value in the negated problem, and 

(11^ 



'AB — \ 2 I AB 



I log2 1 + U-l W i-f 



For a #2SAT problem with a non-zero ratio a of clauses to variables, the number of 
satisfying solutions is exponentially smaller than the number of unsatisfying solutions: 
linv^oo ZjZ = 0. Therefore, we find that in this case — > in the large n limit. 



5. Renyi entropies and the complexity of #SAT 

With these results for the Renyi entropies, we are in a position to test whether or not, 
at least for the #2SAT and its negation, the entropies can predict the degree of difficult 
in solving a problem. 
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Figure 4. Correlation between the singular values associated to W and to the negation 
W for n = 20. For each of 1000 realizations of the problem, the singular values A| for 
the a random 2SAT problem and the Xf, ones associated to the negated problem are 
rank ordered, and the pair (A?, A|) is plotted. The total number of pairs in the plot, 
for all the 1000 realizations combined, is 14827. The dashed straight line is a guide 
to the eye. The slight deviations away from the line at higher values of the singular 
values are dominated by the second largest singular value of the negated problem, the 
one with k = 1. 

• 2 SAT solutions - In the case of #2SAT, we find that both S^ 0) and S iq=2) 
are volumetric, i.e. they scale linearly with n. Since S^ ^ > S^ 1 ^ > S^ q=2 \ the 
entanglement entropy with q — >■ 1 is also volumetric. Because S( 9_> °) is volumetric, 
carrying out the counting exactly cannot be done efficiently, which is expected since, 
after all, #2SAT is ^P-complete and hence unavoidably hard. Thus, in this case, the 
entropy is a good predictor of the difficult of the problem. Moreover, because S^ 9-5-1 ) is 
also volumetric, we do not expect that an approximate counting algorithm should exist 
either [I]. 

• 2SAT non-solutions - In the case of the negation of #2SAT, we find that S^ ^ 
is volumetric, while S^^ 1 ' vanishes for large n. We would then conclude that counting 
the exact number of non-satisfying binary strings is still unavoidably hard. However, 
because S^^ 1 ' — > 0, one can efficiently approximate the number of non- satisfying inputs. 
This results is in agreement with what is known about the problem, as we explain below 
in more detail. 

That counting exactly in one problem is equivalent to counting exactly in the other 
is evident, as Z = 2 n — Z. Thus, if one finds Z one has Z and vice versa. But since 
Z 3> Z, it is easier to find an approximation to Z than to Z within the same relative 
error. The issue can be alternatively stated as follows. A 2SAT problem is presented 
in conjunctive normal form (CNF), with m OR-clauses with two literals, all AND-ed 
together. Its negation is then in disjunctive normal form (DNF), with m AND-clauses 
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with two literals, all OR-ed together. It is known that DNF problems admit a fully 
polynomial-time randomized approximation scheme (FPRAS) jH]. That the entropy 
S^ 1 *) — s. while, in contrast, S^ 1 ^ is volumetric, is the way in which our entanglement 
entropy approach signals that the DNF problem is simpler than the CNF one. 

6. Conclusion 

Let us conclude with a few remarks. First, for CNF problems defined on a graph, one 
may expect that the entanglement should be volumetric. However, it is not always the 
case that a problem defined on graph will have volumetric entanglement, as exemplified 
in the DNF problem that results from the negation of a CNF one. The intuition as 
to why that is so is that the disjunctive form splits the problems into many disjoint 
ones, whereas the conjunctive ones ties the bits together, and is mean-field like. The 
entanglement is one way to capture the fact that some problems cannot be simplified 
by dividing into subproblems, while others can. 

Finally, we would like to mention a possible practical way to explore the 
entanglement as predictor of whether it may or not be possible to approximately count 
for a given problem. Notice that we succeeded in getting the scaling of the Renyi 
entropies even for systems of modest sizes. Therefore, one could get a sense of whether 
a FPRAS may possibly exist, without explicitly constructing one, using an entanglement 
entropy finite-size scaling analysis. 
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Appendix A. Numerical simulations with matrix product states 

We employed the matrix computing method introduced in \X0\ to verify numerically that 
cx n for the conjunctive form of the #2SAT problem. In this method, we associate 
to each binary variable Xj a pair of real matrices Mj* and Mj of dimensions Dj-i x Dj. 
The weight W(x) of each configuration x is written as the trace of a product of these 
matrices, namely, 

W(x h ...,x n )=tr(M* 1 ---M^). (A.l) 

(The trace can be dropped if we consider the first and last matrices to be row and 
column vectors, respectively, namely, Dq = D n = 1.) It is straightforward to show that 
Z, the number of satisfying configurations, is given by the expression 

Z = tr [(M° + Ml) ■ ■ ■ (M° n + Ml)] . (A.2) 

The matrix representation of the weight W(x) allows us treat (Op) similarly to a quantum 
mechanical superposition state. Then, by operating sequentially on adjacent pairs of 
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matrices (Mj^MjQ 1 ), we can simultaneously check the satisfiability of all 2 n instances 
of a CNF problem. 

One starts by setting W = Wo(x) = 1 for all values of the n-bit string x. This 
can be easily implemented by choosing Dj = 1 and Mj = Mj = 1 for all j = 1, . . . , n, 
yielding Z = Zq = 2 n . Each clause Ck in a CNF can eliminate non-satisfying instances 
of the problem and, consequently, reduce Z. We call these gate operations filters. Thus, 
starting from an initial weight distribution W (x), the vector \W ) evolves into a state 
\Wc) by the sequential application of filters that block states that do not satisfy a CNF 
Boolean expression C — C\ A C 2 • • • A C m . Notice that there are four possible types of 
two-bit OR filter gates, depending on whether the input bits are negated or not. 

Since the order in which bits appear in the clauses in C is random and only adjacent 
bit operations are allowed in matrix computing, matrices have to be moved up and 
down the bit string. This is done through SWAP gates [10] . These SWAP gates, when 
combined with other logic gates such as OR, tend to rapidly increase the rank of the 
matrices. This is a limiting factor of the method, as every gate operation between bits 
(j — l,j) employs a singular value decomposition which requires 0{Dj_^) floating point 
operations. On the other hand, the filters, by state elimination, partially compensate 
the growth in matrix rank and -D max — max{.Dj} tend to peak around m ~ n. Therefore, 
for each realization of a CNF, the computational cost of the numerical calculation scales 
as 0(DIJ. 

In order to minimize the number of SWAP operations and limit the growth of 
matrix ranks, we employ two pre-processing strategies. First, we reorder the bits in the 
string so that the sum of pair-wise distances between bits in the clauses of the CNF is 
minimized. This is done through a Monte Carlo sampling with a 100% rejection rate if 
the new configuration has a higher total distance than the previous one. Typically, 30 
sampling steps are used. 

Second, we reorder the clauses so that those bits participating in few or no clauses 
are acted on first. Bits which are no longer required are set to an inactive state by 
absorbing their bit matrices into matrices of a neighboring bit and replacing their 
matrices with identity ones. These measures reduce the matrix ranks substantially, 
which in turn allow us to count exactly all solutions for large CNFs. We find empirically 
that (-Dm ax ) ~ 2 01n . This favorable scaling (as compared to the computational cost 
of the best known algorithm for solving exactly the #2SAT problem, which scales as 
2°- 329n [?]) corresponds to an average behavior and does not apply to hard, worst-case 
realizations of the CNF. 

Appendix B. Bit-string partitioning 

Once the complete sequence of m clauses of a CNF is implemented with matrix 
computing, the partition matrix W can in principle be obtained in the following way. 
Consider the bit-string matrix set {Mj 3 } resulting from the sequence of Boolean gates. 
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Following the definition of W, we can write 



D 



n /2 



y^x A ;x B — A XA ;a B a - XB , (B-l) 



a=l 



where 



A xl ... Xn/r ,a =M^...M x n %> (B.2) 

Ba; Xn/w ... Xn =K%+l---K n - (B.3) 

This decomposition of W can be used to compute some Renyi entropies without the 
need to obtain W explicitly. While it is straightforward to show that S^ ^ = log 2 D n / 2 , 
5'(9> 1 ) requires an additional manipulation. Combining the singular value decomposition 
W = U A V T and with (IB. 11) . we can write 

\ 2 k q = tr (A 2? ) = tr [{U T WW T U) q ] = tr [(BB T A r A) q ] , (B.4) 

fc=i 

where we used the cyclicity of the trace. Finally, combining (|6]) with (IB. 41) . one finds 

The two major advantages in this approach as compared to performing the singular 
values decomposition of W is the reduced number of floating point operations (since no 
singular value decomposition of W is needed) and the reduced memory allocation for 
matrix storage. The disadvantage is that one cannot compute S^ 1 ^. In practice, this 
is the only viable numerical approach that we know for finding the scaling of with 
n when n > 20. 

Appendix C. Heuristic argument for ( I8a|l86|) 

Because W(x) = 1 — W(x), the associated matrix W = M — W, where W is the 2 l x 2 n ~ l 
matrix constructed in the main text and Ai is a matrix of the same size with all entries 
equal to 1. It follows that 

WW T = MM T - WM - MW T + WW T 

= 2 n - l H-WM- MW T + WW T , (C.l) 

where we defined the 2 l x 2 l matrix % with all entries equal to 1. Now, 

k=l k=l q=l k=l 

where we used that the average over the lines i and j of the matrix W should become 
independent of the line index for large enough matrices. We thus have that 

WW T w (2 n - 2Z) -V. + WW T . (C.3) 
2 l 

The matrix H has one eigenvalue equal to 2 l , and 2 l — 1 zero eigenvalues. Therefore, 
(2 n — 2Z) ^-"H alone has one eigenvalue Aq = 2 n — 2Z, and 2 l — 1 zero eigenvalues. 
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Let us discuss the situation when Z < 2 n /2, in which case Aq = 2 n — 2Z > 0. One 
can then add WW T as a perturbation, which lifts up the massive degeneracy without 
affecting much the non-zero eigenvalue because of the large splitting. Therefore, the 
non-zero eigenvalues of WW should be all the non-zero eigenvalues A^ of WW T , plus 
the extra large Aq eigenvalue. This is the heuristic argument for ( ]5of86j) . which we 
support numerically as explained in the text. 

We remark that we arrived at the result that the negated problem with weights 
W(x) = 1 — W(x) has one more singular value than the problem with weights W(x) 
using that the number of solutions of W (x) = 1 is less than the number of non-solutions 
W(x) = 0, i.e. Z < 2 n /2. This ensured that the Aq = 2 n — 2Z singular value was 
positive; if it were negative, the perturbative argument should break down, for one 
would have to restore the positiveness of all the singular values in the end. Therefore 
what determines which of the two problems, with weights W(x) or W(x), has more 
singular values is which one has more solutions. That we used Z < 2 n /2 breaks the 
symmetry between the two problems, and explains why we cannot use the argument 
twice doing a double negation W(x). The argument only goes in one direction, and 
applies only to the case when Z < Z. 
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